On Learning and Covering Structured Distributions

We explore a number of problems related to learning and covering structured distributions. Hypothesis Selection: We provide an improved and generalized algorithm for selecting a good candidate distribution from among competing hypotheses. Namely, given a collection of N hypotheses containing at least one candidate that is ε-close to an unknown distribution, our algorithm outputs a candidate which is O(ε)-close to the distribution. The algorithm requires O(logN/ε) samples from the unknown distribution and O(N logN/ε) time, which improves previous such results (such as the Scheffé estimator) from a quadratic dependence of the running time on N to quasilinear. Given the wide use of such results for the purpose of hypothesis selection, our improved algorithm implies immediate improvements to any such use. Proper Learning Gaussian Mixture Models: We describe an algorithm for properly learning mixtures of two single-dimensional Gaussians without any separability assumptions. Given ?̃?(1/ε) samples from an unknown mixture, our algorithm outputs a mixture that is ε-close in total variation distance, in time ?̃?(1/ε). Our sample complexity is optimal up to logarithmic factors, and significantly improves upon both Kalai et al. [40], whose algorithm has a prohibitive dependence on 1/ε, and Feldman et al. [33], whose algorithm requires bounds on the mixture parameters and depends pseudo-polynomially in these parameters. Covering Poisson Multinomial Distributions: We provide a sparse ε-cover for the set of Poisson Multinomial Distributions. Specifically, we describe a set of n 3)(k/ε)poly(k/ε) distributions such that any Poisson Multinomial Distribution of size n and dimension k is ε-close to a distribution in the set. This is a significant sparsification over the previous best-known ε-cover due to Daskalakis and Papadimitriou [24], which is of size n, where f is polynomial in 1/ε and exponential in k. This cover also implies an algorithm for learning Poisson Multinomial Distributions with a sample complexity which is polynomial in k, 1/ε and log n.